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Abstract 

Consider a pair of correlated Gaussian sources {Xi,X2)- Two separate encoders observe the two 
components and communicate compressed versions of their observations to a common decoder. The 
decoder is interested in reconstructing a Unear combination of Xi and X2 to within a mean-square 
distortion of D. We obtain an inner bound to the optimal rate-distortion region for this problem. A 
portion of this inner bound is achieved by a scheme that reconstructs the linear function directly rather 
than reconstructing the individual components Xi and X2 first. This results in a better rate region 
for certain parameter values. Our coding scheme relies on lattice coding techniques in contrast to more 
prevalent random coding arguments used to demonstrate achievable rate regions in information theory. 
We then consider the case of linear reconstruction of K sources and provide an inner bound to the optimal 
rate-distortion region. Some parts of the inner bound are achieved using the following coding structure: 
lattice vector quantization followed by "correlated" lattice-structured binning. 

1 Introduction 

Since its inception in 1973 by Slepian and Wolf, the problem of distributed source coding has been a source 
of inspiration for information/communication/data-compression theory community because of its formidable 
nature (in its full generality) and its wide scope of practical applications. In this problem, a collection of K 
correlated information sources, with ith source having an alphabet Afj, is observed separately by K encoders. 
Each encoder maps its observations into a finite- valued set. The indices from these sets are transmitted 
over K noiseless but rate-constrained channels to a joint decoder. The decoder is interested in obtaining L 
reconstructions with L fidelity criteria (one for each). The ith reconstruction has an alphabet Xi, and the 
ith fidelity criterion is a mapping from the product of alphabets of a subset of the sources and to the set 
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of nonnegative real numbers. The goal is to find a computable performance limit for this communication 
problem. The performance limit, also referred to as the optimal rate-distortion region, is expressed as the 
set of all {K + _L)-tuples of rates of the K indices transmitted by the encoders and distortions of the L 
reconstructions of the decoder that can be achieved in the usual Shannon sense. 

Toward this goal, progress has been made in a number of directions. In the following we restrict our 
attention to the case of the collection of stationary memoryless sources. In [1], a solution to the problem 
was given for the case when the decoder wishes to reconstruct all the sources losslessly. In [31 3] , the case 
of lossless "one-help-one" problem was resolved. Here the decoder wishes to reconstruct only one of the 
source £]losslessly {K = L+l = 2). In [5], the case of lossy "one-help-one" problem was resolved for the case 
when the rate of the helper is greater than its entropy (also referred to as the Wyner-Ziv problem). In [51 [7], 
an inner bound, and an outer bound (also known as the Berger-Tung inner and outer bounds respectively) 
to the performance limit arc given for the case where (a.) K = L = 2 and (b) the fidelity criterion of each 
source does not depend on the other source (also referred to as independent fidelity criteria). In [8], an inner 
bound to the performance limit is given for the case of lossy "one-help-one" problem. In [9J, an inner bound 
to the performance limit is given for the case when the decoder wishes to reconstruct a function of K sources 
losslessly. It was also shown that this is optimal for the case when the sources are conditionally independent 
given the function. In jlOj . the performance limit is given for reconstructing losslessly the modulo-2 sum of 
two binary correlated sources, and was shown to be tight for the symmetric case. This has been extended 
to several cases in |12| (see Problem 23 on page 400) and |14| . An improved inner bound was provided for 
this case in [15] . The key point to note is that the performance limits given in [TUl El [H] are outside the 
inner bound given in [9]. In |16j . the performance limit is given for the case where (a) K = L = 2, (b) one 
of the sources is reconstructed losslessly and the other with a independent fidelity criterion. In [18] (also see 
[TTl [T71 \W[ EUl \n\ m]), an inner bound to the performance limit of the CEO problem]^ was given. This 
problem for the quadratic Gaussian case essentially boils down to reconstructing a certain linear function of 
the sources with mean squared error fidelity criterion. It was shown that this inner bound is tight for some 
cases in [25l|30]. For the vector Gaussian CEO problem, inner and outer bounds were derived in [27l [28]. 
These bounds were shown to be tight under some conditions. In |31j . the performance limit is given for the 
case of lossless reconstruction of a function of two sources with the rate of one of the sources being greater 
than or equal to its entropy. The lossy version is addressed in |32l I33j . Regarding the Berger-Tung inner 
bound, it was shown that this is tight for (a) the high-resolution case with independent fidelity criteria in 
|44j . (b) the jointly Gaussian case K = 2, L = 1 and independent squared error fidelity criterion in [21], 
and (c) the jointly Gaussian case with if = 2, L = 2 and independent squared error criteria in [3S]. In 
|35j . it was also shown that a Berger-Tung based coding scheme is optimal for the case of reconstruction of 
certain linear functions of two jointly Gaussian sources with squared error criterion. A general outer bound 

-"^The source which docs not enter into any of the fidelity criteria is referred to as a helper. When the rate at which the 
helper is transmitted is greater than its entropy, the helper is also referred to as side information. 

^This is a variant of the general distributed source coding problem mentioned above. This is closely related to another class 
of distributed source coding problems known as remote source coding problems. Here the encoders observe a noisy version of 
the sources. However it can be shown using the techniques of [221123) that the remote source coding problems are equivalent to 
a class of general distributed source coding problems mentioned above. 
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to the performance limit of the general distributed source coding problem was given in f29]. In [34], the 
performance limit was given for the lossy "one-help-many" problem with independent fidelity criteria and 
the sources being conditionally independent given the helper which is transmitted at a rate greater than its 
entropy. In |26j . the performance limit was given for the quadratic jointly Gaussian lossy "many-help-one" 
problem with the condition that the helpers are conditionally independent given the source. In [36], the 
performance limits were obtained for the case of quadratic Gaussian "many-help-one" problem where the 
sources satisfy a "tree-structure" . In [37] , the performance limit is given for the case where one of the 
sources needs to be reconstructed with an independent fidelity criterion and the rest of the sources need to 
be reconstructed losslessly. In [38], infinite order descriptions were provided for the performance limits of 
the general case of two terminal source coding problem {K — 2) with independent distortion criteria. This 
was extended to the case of more than two sources in [55] . 

With regard to above set of results, we would like to make the following observations, (a) Most of the above 
approaches, except that of [10] and its extensions in [El [HI US], use random vector quantization followed 
by independent random binning (see Chapter 14 of [T^) of the quantizer indices, (b) The four exceptions, 
which consider only lossless source coding problems, deviate from this norm, and instead use structured 
random binning based on linear codes on finite fields. Further, the binning operation of the quantizers of 
the sources are "correlated" . This incorporation of structure in binning appears to give improvements in the 
rates especially for those cases that involve reconstruction of a function of the sources. Moreover, it is still 
not known whether it is possible to approach this performance without the structured codes, (c) For some 
distributed source coding problems (that belong to the first category) , whose performance limits were derived 
using random coding and random binning, it is well-known that these limits can also be approached using 
structured codes. For example structured codes were considered for (a) the Slepian- Wolf problem in [40], (b) 
the Wyner-Ziv problem for the binary case with Hamming distortion and for the quadratic Gaussian case in 
|45| . (c) the Berger-Tung inner bound for the two terminal quadratic Gaussian problem with independent 
fidelity criteria in [45] and (d) high-resolution distributed source coding problem with independent fidelity 
criteria in |44j . 

With this as a motivation, in this paper we consider a lossy distributed source coding problem with 
K jointly Gaussian sources with one reconstruction, i.e., L = I. The fidelity criterion has the additional 
structure that is given by the following. The decoder wishes to reconstruct a linear function of the sources 
with squared error as the fidelity criterion. We consider a coding scheme with the following structure: sources 
are quantized using structured vector quantizers followed by "correlated" structured binning. That is, the 
binning operations of the quantizers of the sources are not performed "independently" . The structure used 
in this process is given by lattice codes. We provide an inner bound to the optimal rate-distortion region. 
We show that the proposed inner bound is better for certain parameter values than an inner bound that can 
be obtained by using a coding scheme that uses random vector quantizers following by independent random 
binning. For this purpose we use the machinery developed by [ITJ H^J US] |3S1 112 foi' the Wyner-Ziv problem 
in the quadratic Gaussian case. 

We also beheve that the proposed scheme can be used as a building block to provide an inner bound to 
the optimal rate-distortion region for the case when the decoder wishes to reconstruct all the sources with 
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independent squared error fidelity criterion. This will be addressed in our future work. The rest of the paper 
is organized as follows. Rather than giving the main result for the most general case first and then considering 
special cases, we first consider the case of two sources and obtain the result and then provide the result for 
the general case. In Section |2] we give a concise overview of the asymptotic properties of high-dimcnsional 
lattices that are known in the literature and we use these properties in the rest of the paper. In Section 
[3j we define the problem formally for the case of two sources and present an inner bound to the optimal 
rate-distortion region given by a coding structure involving structured quantizers followed by "correlated" 
structured binning. Further, we also present another inner bound achieved by a scheme that first obtains 
a lossy reconstruction of the sources, and then obtains a reconstruction of the linear function. The latter 
scheme is based on the Berger-Tung inner bound. An overall achievable rate region can be obtained by 
combining these two schemes. Then we present our lattice based coding scheme and prove achievability of 
the inner bound. We also provide motivation and intuition about the proposed coding scheme in this section. 
In Section |4] we consider a generalization of the problem that involves reconstruction of a linear function 
of an arbitrary finite number of sources. We also demonstrate how the general solution simplifies in certain 
special cases. In Section [5] we provide a set of numerical results for the two-source case that demonstrate 
the conditions under which the lattice based scheme performs better than the Berger-Tung based scheme. 
We conclude with some comments in Section [6l 

A word about the notation used in this paper is in order. Let /(•) be an arbitrary function that takes 
as input a scalar. Then the function /"(■) takes an n-length vector as input and operates component- 
wise on the components of that vector. This notation generalizes to functions of more than one variable 
as well. Variables with superscript n denote an n-length random vector whose components are mutually 
independent. However, random vectors whose components are not independent are denoted without the use 
of the superscript. The dimension of such random vectors will be clear from the context. 

2 Preliminaries on high-dimensional Lattices 
2.1 Overview of Lattice Codes 

Lattice codes |54j play the same role in Euclidean space that linear codes play in Hamming space. Introduc- 
tion to lattices and to coding schemes that employ lattice codes can be found in [321 113 IHl IMl ES] . Lattice 
codes have been used in other related multiterminal source coding problems in the literature |56l |57l |58l |59j 
IBU] . In the rest of this section, we will briefiy review some properties of lattice codes that are relevant to 
our coding scheme. We start by defining various quantities of interest associated with lattices. We use the 
same notation as in jJS] for these quantities. 

An n-dimensional lattice A is composed of all integer combinations of the columns of an n x n matrix G 
called the generator matrix of the lattice. 

K = {leW : l = G-i for some i e Z"} (1) 

Associated with every lattice A is a natural quantizer namely one that associates with every point in M" its 



4 



nearest lattice point. This quantizer can be described by the function 

QA{x) = le A where \\x''l\\<\\x~l\\ for all f e A. (2) 

The quantization error associated with the quantizer Qa(') is defined by 

X mod A = x — Qa{x). (3) 

The basic Voronoi region of a lattice A is the set of all points closer to the origin than to any other lattice 
point, i.e., 

Vo(A) = {xeR" : Qa{x) = 0"} (4) 

where 0" is the origin of M". The second moment of a lattice A is the expected value per dimension of the 
norm of a random vector uniformly distributed over Vo(A) and is given by 

1 A) /A^ II X IP dx 

Let the normalized second moment be give by 

- V^) 

where V{A) = J^g(^A) When used as a channel code over an unconstrained AWGN channel with noise Z 
having variance cr^ let the probability of decoding error be denoted by 

P,(A,4) = Pr(Z"^Vo) (7) 

where is the random noise vector of length n. 

The mod operation defined in equation (|3| satisfies the following useful distributive property. 

{{x mod A) + y) mod A = {x + y) mod A Vx, y. (8) 

It is known (see [H] [IB]) that the quantization error of a lattice quantizer A can be assumed to have 
a nearly uniform distribution over the fundamental Voronoi region Vo of the quantizer. This assumption is 
completely accurate in the case of subtractive dithered quantization where a vector uniformly distributed 
over Vo (called the dither) is added at the encoder before quantization and subtracted at the decoder. It 
has been shown in |42| that for an optimal lattice quantizer, this noise is wide-sense stationary and white. 
Further, as the lattice dimension n oo, for optimal lattice quantizers, the quantization noise approaches 
a white Gaussian noise process in the KuUback-Leibler divergence sense. 

Lattices have been studied extensively for efficient packing and covering. A systematic study of lattice 
packings was initiated by Minkowski in [48j . where existence of good lattice packings was shown. A formal 
study of lattice covering appears to have been initiated by Kershner in [30] ■ See [5T] for a thorough review of 
existence of efficient lattice packings and coverings. Lattice codes have been employed in the point-to-point 
setting for quantization of Gaussian sources with squared error fidelity criterion and also in coding for the 
AWGN channel with power constraint. In [45 , the existence of high dimensional lattices that are "good" 
for quantization and for coding is discussed. The criteria used therein to define goodness are as follows: 
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• A sequence of lattices A^") (indexed by the dimension n) is said to be a good channel cr|-code sequence 
if Ve > 0, there exists N{e) such that for all n > N{e) the following conditions are satisfied: 

T^(A''"') < 2"'3 logC^'recrlj+e)^ (9) 

Pe(A("\a|) < 2-"-^(^) (10) 

for some E{e) > 0. 

• A sequence of lattices A^") (indexed by the dimension n) is said to be a good source D-code sequence 
if Ve > 0, there exists N{£) such that for all n > N{e) the following conditions are satisfied: 

log(27reG(A("))) < e (11) 

ct2(A(")) ^ D. (12) 



2.2 Nested Lattice Codes 

For lossy coding problems involving side- information at the encoder/decoder, it is natural to consider nested 
codes. Wyner proposed an algebraic binning approach involving linear codes for the Slepian-Wolf problem 
[2]. Adapting this scheme to the case of lossy coding, nested codes for the Wyner-Ziv problem were proposed 
in 133]. We review the properties of nested lattice codes briefly here. Further details can be found in jl5] . 

A pair of n-dimensional lattices (Ai, A2) is nested, i.e., A2 C Ai, if their corresponding generating matrices 
Gi,G2 satisfy 

G2 = Gi • J (13) 

where J is an n x n integer matrix with determinant greater than one. Ai is referred to as the fine lattice 
while A2 is the coarse lattice. The points of the set 

{Ai mod A2} = {Ai n Vo,2} (14) 

are called the coset leaders of A2 relative to Ai . The nesting ratio of this nested lattice is defined as '■{/V2/V1 
where Vi = V{Ai) is the volume of the Voronoi region of lattice A^, i — 1,2. 

In many applications of nested lattice codes, we require the lattices involved to be a good source code 
and/or a good channel code. We term a nested lattice (Ai, A2) good if (a) the fine lattice Ai is both a good 
(5i-source code and a good Ji-channel code and (b) the coarse lattice A2 is both a good 52-source code and a 
i52-channel code. For such a nested lattice code (Ai, A2), the number of coset leaders of A2 relative to Ai is 
about {62/61)""^^. A code employing the coset leaders as codewords would thus have a rate of | log(^2/'5i). 
Equivalently, the rate of such a code is the logarithm of the nesting ratio of the nested lattice (Ai, A2). 

The existence of good lattice codes and good nested lattice codes (for various notions of goodness) has 
been studied in [46j |47j which use the random coding method of |49l|52]. In |47], it was shown that there 
exists lattices which are simultaneously good in both the source and channel coding senses described above. 
In [35] , the existence of nested lattices where the coarse lattice is simultanously good as a source and channel 
code and the fine lattice is a good channel code was proved. 
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3 Distributed source coding for the two-source case 



3.1 Problem Statement and Main Result 

In this section we consider a distributed source coding problem for the case of two sources Xi and X2. The 
function to be reconstructed at the decoder is assumed to be the hnear function Z ^ F{Xi,X2) = Xi — 0X2 
unless otherwise specified. Consideration of this function is enough to infer the behavior of any linear 
function ciXi + C2X2 and has the advantage of fewer variables. We consider the more general case of 
F{Xi, ...,Xk) = Eti c^X^ in Section Q 

We define the coding problem formally below. Consider a pair of correlated jointly Gaussian sources 
{Xi,X2) with a given joint distribution PxiX2{^ij^2)- The source sequence {X",X2) is independent over 
time and has the product distribution ]Xi^iPXiX2{^iiTX2i)- Consider the following average squared error as 
the fidelity criterion: d : M" x M" ^ M+ given by 

n 

d(x",y") = -V(x,-2/,)2. (15) 
n ^-^ 

i=l 

Definition 3.1. Given such a jointly Gaussian distribution pxiX2 ^"^d a distortion function d{-,-) a trans- 
mission system with parameters (n, 01,6*2, A) is defined as the set of mappings 

/, :M"^{l,2,...,0a /orz=l,2 (16) 
5:{l,2,...,0i}x{l,2,...,02}^M" (17) 

such that the following constraint is satisfied 

E{d{F^X^,X^),g{f,{X^)j2{X^m < A. (18) 

We say that a tuple (Ri, R2, D) is achievable if^e > 0, 3 for all sufficiently large n, a transmission system 
with parameters (n,0i,02, A) such that 

-\oge,<R, + e for I =1,2 
n 

A<D + e. 

The performance limit is given by the optimal rate-distortion region which is defined as the set of all achievable 
tuples {Ri, R2, D) . This problem is graphically illustrated in Fig. [7] 

Without loss of generality, the sources can be assumed to have unit variance and let the correlation 
coefficient p > 0. For the rest of this section, these assumptions are made unless otherwise stated. 

One possible coding scheme for this problem would be the following. The decoder reconstructs lossy 
versions (^1,^2) of the sources (Xi, X2) and uses the best estimate of Z given (Wi, WK2) as the reconstruction 
Z. The rate region for such a scheme can be derived using the Berger-Tung inner bound [6l[7]. One of the 
main result in this paper is to show that for certain parameter values, there exists a better coding scheme 
that enables the decoder to reconstruct Z directly without resorting to reconstructions {Wi, ^2)- We present 
the rate region of our scheme below. 
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Figure 1: Schematic Illustration of the problem 



Theorem 3.1. The set of all tuples of rates and distortion (Ri, R2, D) that satisfy 



'2Ri 



-2R2 



< 



D 



(19) 



achi 



levable. Here, — 1 



2pc is the variance of the function Z to be reconstructed. 



Proof: See Section |321 

We also present another achievable rate region based on ideas similar to the Berger-Tung coding scheme 
[B] [7]. From here on, we shall refer to this rate region as the Berger-Tung based rate region and the scheme 
that achieves this as the Berger-Tung based coding scheme. 



Theorem 3.2. Let the region TZVin be defined as follows. 



(-3i,92)eRi 
Ri 

where a = 1 — and K 



{Ri,R2,D}: Ri > - log — r , 

2 qi(l + q2) 



R: 



- 2 °S 92(1 + qi) 



2 gi92 



D > 



qia 



q2C?a - 



(l + gi)(l + g2)-pM ■ ^'^^^ 
is the set of positive reals. Then the rate distortion tuples (Ri, R2, D) which belong 



to 'JZ'D*n are achievable where * denotes convex closure. 

Proof: Follows directly from the application of Berger-Tung inner bound with the auxiliary random 
variables involved being Gaussian. 

In many distributed source coding problems involving jointly Gaussian sources ( |25l 1301 135j). the use of 
Gaussian auxiliary random variables results in the optimal or largest known rate region. It was conjectured 
in [5J [7] that choosing the auxiliary random variables to be Gaussian indeed results in the optimal rate 
distortion region for the problem of reconstructing both sources with independent distortion criteria. This 
was shown to be true in [35j . With this as motivation, we have used Gaussian auxiliary random variables to 
derive an inner bound to the performance limit of this problem based on the Berger-Tung coding scheme. 

We have the following lemma that gives the minimum sum rate of the second approach which will be 
used in later sections for comparing the performance limits given by the above two theorems. 
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Lemma 3.1. For a given distortion D, the minimum sum rate Rsum = Ri + R2 that lies in the region TZ'D*^ 
of Theorem \3.S\ is given by the lower convex envelope of the following region. 

1, Ac(ac-pD) ^ . ( 2ac 2ac^ 1 

Rsura>\\0g{^^^^) al>D>^,c<l (22) 

Rsu,n >^log{ \ " al>D> c > 1 23 

2 \ D — a J p + c 

Rsn,n = D>al (24) 
Proof: This derivation is detailed in Appendix [A] 

For certain values of p, c and D, the sum-rate given by Theorem |3.1| is better than that given in Theorem 
|3.2[ This implies that each rate region contains rate points which are not contained in the other. Thus, 
an overall achievable rate region for the coding problem can be obtained as the convex closure of the union 



of all rate distortion tuples {Ri, R2, D) given in Theorems 3.1 and 3.2 A further comparison of the two 
schemes is presented in Section |5] Note that for c < 0, it has been shown in that the rate region given 
in Theorem |3.2|is tight. 



3.2 The Coding Scheme 

In this section, we present a lattice based coding scheme for the problem of reconstructing the above linear 
function of two jointly Gaussian sources whose performance approaches the inner bound given in Theorem 



3.1[ In what follows, a nested lattice code is taken to mean a sequence of nested lattice codes indexed by 
the lattice dimension n. 

We will require nested lattice codes (Aii,Ai2,A2) where A2 C An and A2 C A12. We need the fine 
lattices An and A12 to be good source codes (of appropriate second moment) and the coarse lattice A2 to 
be a good channel code. The proof of the existence of such nested lattices is detailed in Appendix [B] where 
we show the existence of a nested lattice (An, A12, A2) such that An C A12 C A2 or A12 C An C A2 and all 
three lattices are good source and channel codes simultaneously. The parameters of the nested lattice are 
chosen to be 

^'(An) = 91 (25) 
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a2(Ai2) = ^^-gi. (26) 



<^^(A2) = (27) 

where < qi < Da^/{a^ — D). The coding problem is non-trivial only for D < a\ and in this range, 
£'a|/(CT| — D) < o'^(A2) and therefore A2 C An and A2 C A12 indeed. Note that the order of nesting 
between the lattices An and A12 depends on whether qi > L)cr|/2(o'| — D) or not. However, this is 
irrelevant for the proof which only requires A2 C An and A2 C A12. 
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Figure 2: Distributed coding using lattice codes to reconstruct Z = X\ — cXi 



Let U\ and \Ji be random vectors (dithers) that are independent of each other and of the source pair 
{Xx,X-2). Let Ui be uniformly distributed over the basic Voronoi region Vo,ii of the fine lattices Ai^ for 
i= 1,2. The decoder is assumed to share this randomness with the encoders. The source encoders use these 
nested lattices to quantize X\ and 0X2 respectively according to equation 

= (QAii(^r + t^i)) mod A2, (28) 
^2 = (Qai,(cX2" + C/2)) mod A2. (29) 

Note that the second encoder scales the source Xi before encoding it. The decoder receives the indices S\ 
and iS'2 and reconstructs 



([(5i-C/i)-(52-t/2)]modA2). (30) 



'z 

This coding scheme is illustrated in Fig. |2] The rates of the two encoders are given by 

1 

2 ^""91 (4-^) 



^1= o log 777^2^^ (31) 



Clearly, for a fixed choice of q\ all rates greater than those given in equations (31 1 and (32 1 are achievable. 
The union of all achievable rate-distortion tuples {Ri, R2, D) over all choices of qi gives us an achievable 
region. Eliminating qi between the two rate equations gives us 



which is the rate region claimed in Theorem 3.1 It remains to show that this scheme indeed reconstructs 
the function Z to within a distortion D. We show this in the following. 

Using the distributive property of lattices described in equation (|8| , we can reduce the coding scheme to 
a simpler equivalent scheme by eliminating the first mod-A2 operation in both the signal paths. This results 
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-<?2 



Figure 3: Equivalent representation of Fig. [2] 



in an equivalent representation of the coding scheme as shown in Fig. [3] The decoder can now be described 
by the equation 



Z = 



ol-D 



([(Xr + e,J - (cX^" + e,,)] mod A2) 



mod A2) 



where e^^ and are dithered lattice quantization noises given by 

e,, = gA,,(xr + f/i)-(xr + [/i), 

e,. = QK.A^X'i + (72) - (CX2" + C/2). 



(34) 
(35) 



(36) 
(37) 



The subtractive dither quantization noise e,. is independent of both sources X\ and Xi and has the same 
distribution as —Ui for z = 1,2 [45] . Since the dithers U\ and J72 are independent and for a fixed choice of 
the nested lattice e^^ is a function of Ui alone, e^^ and e^^ are independent as well. 



Let e„ = e 



91 



be the effective dither quantization noise. The decoder reconstruction in equation ( 35 1 



can be simplified as 



al-D 



i[Z" + eg] mod A2) 



^ z'- 



(Z- + e,) 



D 



N. 



(38) 

(39) 

(40) 
(41) 



c.d . 



The = in equation (39) stands for equality under the assumption of correct decoding. Decoding error 



occurs if equation (39 1 doesn't hold. Let Pg be the probability of decoding error. Assuming correct decoding. 



the distortion achieved by this scheme is the second moment per dimensioij^ of the random vector N in 

^We refer to this quantity also as the normahzcd second moment of the random vector A'^. This should not be confused with 
the normalized second moment of a lattice as defined in equation |6j|. 
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equation (41). This can be expressed as 

T?. - n\ ^ F, II P. l|2 / n \ ^ F. II z" ||2 

(42) 



E II iV ||2 _ /a| E II e, f fDVE 



'Z / "- v^l 

where we have used the independence of e^j and e^^ to each other and to the sources Xi and X2 (and 
therefore to Z = Xi — 0X2). Since Cg^ has the same distribution as —Ui, their expected norm per dimension 
is just the second moment of the corresponding lattice cr^ (Ai^). Thus the effective distortion achieved by the 
scheme is 

= D. (43) 
Hence, the proposed scheme achieves the desired distortion provided correct decoding occurs at equation 



( 39 1 . Let us now prove that equation ( 39 1 indeed holds with high probability for an optimal choice of the 



2 ""^ 



E II Z" + eg f _ E II Z" ||2 ^ E II -Ui - U: 
n n n 



nested lattice, i.e., there exists a nested lattice code for which Pg — > as n ^ 00 where, 

Pe - Pr ((Z" + e,) mod A2 ^ (Z" + ej) . (44) 
To this end, let us first compute the normalized second moment of (Z" + e^). 

(45) 

91 + l^^r^ - 91 (46) 

= ^^=a\A2). (47) 

It was shown in |42| that as n —* 00, the quantization noises e^. tend to a white Gaussian noise for an 
optimal choice of the nested lattice. The following lemma states that Cg also converges in the same way. 

Lemma 3.2. // the two independent subtractive dither quantization noises Cq. tend to a white Gaussian 
noise of the same variance as Cq. in the Kullback-Leibler divergence sense, then Cq = Cq-^ — also tends to 
a white Gaussian noise of the same variance as Cq in the divergence sense. 

Proof: The proof of convergence to Gaussianity of Cq is detailed in Appendix ICl 



We choose A2 to be an exponentially good channel code in the sense defined in Section 2.1 (also see 



[45]). For such lattices, the probability of decoding error Pg in equation (44) goes to exponentially fast 
if {Z" + Cq) is Gaussian. The analysis in [46 also showed that if (Z" + e^) tends to a white Gaussian 
noise vector, the effect on Pg of the deviation from Gaussianity is sub-exponential. Hence, the overall error 
behavior is asymptotically the same as the behavior if (Z" + e^) were Gaussian, i.e., Pg as n — > 00. 
This implies that the reconstruction error Z" — Z tends in probability to the random vector N defined in 



equation (41 1. Since all random vectors involved have finite normalized second moment, this convergence 



in probability implies convergence in second moment as well. Thus the normalized second moment of the 



reconstruction error tends to that of N which is shown to be D in equation ( 43 1 . Averaged over the random 
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dithers Ui and U2, we have shown that the appropriate distortion is achieved. Hence there must exist a pair 



of deterministic dithers that also achieve the given distortion. Combining equations (33) and (43 1, we have 
proved the claim of Theorem |3.1[ 

Remark: Instead of focussing on the entire rate region, if one is interested in minimizing the sum rate 
of the encoders, then it can be checked that the optimal choice of lattice parameters is (t^(Aii) = a^{Ai2) — 
\ ^"-D • case, we require only one nested lattice (Ai, A2) with both encoders using the same nested 

lattice for encoding. 



3.3 Intuition about the Coding Scheme 

In this section, we outline some arguments that justify our choice of lattice codes and the scaling constants 
described in the previous subsection. Our use of lattice codes is motivated by the following. Suppose there 
exists a centralized encoder that has access to both sources X\ and Xi- Clearly, the optimal encoding 
strategy then would be to compute Z ~ X\~ 0X2, and compress it using an encoder, say /(•), that achieves 
the optimal rate distortion function of a Gaussian source of variance ct^- Such a centralized coding scheme 
can be adapted to a distributed setting if the encoder /(•) distributes over the linear function Xi — 0X2- For 
then, from the decoder's perspective, there is no distinction between the centralized and distributed coding 
scheme since 

f{X^-cX2) = .f{X^)- f{cX2). (48) 



A lattice code satisfies the functional form mentioned in equation ( 48 1 and is known to achieve the optimal 



rate distortion function for Gaussian sources. Hence it is an ideal candidate for use as the source encoder. 



The parameters of the lattice code as given in equations (25) and (26) can be justified as below. Without 
loss of generality, let the second source alone be scaled by an arbitrary constant rj. Let the fine lattices 
in the signal path of the two sources have second moments qi = (T^(Ai_i) for i — 1,2. For the case of 
optimal lattices in high enough dimensions, one can think of quantization using the fine lattices A^^i, i = 1,2 
as simulating an AWGN channel of noise variance g^. Such a statement can be made precise by analysis 
similar to the one carried out in the previous subsection. Let Qi^i = 1,2 be N{Q,qi) random variables that 
are single-letter asymptotic equivalents of the subtractive dither quantization noises e,. encountered in the 
previous subsection. 

Referring to the equivalent coding scheme represented in Fig. [3] we see that it suffices to choose the 
coarse lattice A2 to be a good AWGN channel code of second moment equal to 

a2(A2) = Var(Xi + Qi - {7^X2 + Q2)) 

= l + r]^ -2r]p+qi + q2. (49) 

Using the distributive property of lattices (equation ([s]) ) , this scheme can be converted to the one represented 
by Fig. [I 

The rates achieved by this scheme are given by 

Ri = - log for z = 1, 2 (50) 

^ Qi 
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This region can be optimized over all choices of rj subject to an appropriate distortion constraint. It turns 
out that the scaling chosen in Section 3.2 is the optimal choice. The details are described (for the more 
general K user case) in Appendix [Pj 



4 Distributed source coding for the K source case 

In this section, we consider the case of reconstructing a linear function of an arbitrary number of sources. 
In the case of two sources, the two strategies used in Theorems |3 . 1 1 and |3 . 2 1 were direct reconstruction of the 
function Z and estimating the function from noisy versions of the sources respectively. Henceforth, we shall 
refer to the coding scheme used to derive Theorem 3.1 as lattice binning and that used in Theorem 3.2 as 
random binning. 

In the presence of more than two sources, a host of strategies which are a combination of these two 
strategies become available. For example, in the case of 3 sources, one possible strategy would be for all 
users to use the lattice binning while another strategy would be for users 1 and 2 to use lattice binning and 
user 3 to employ random binning. The union of the rate-distortion tuples achieved by all such schemes gives 
an achievable rate region of the problem. 

When a combination of the two strategies are used among the K sources, the order of decoding at the 
decoder becomes significant. The indices which are decoded earlier can be used as side information for 
the indices which are to be decoded later. This raises the question of how to adapt the coding schemes 
of lattice binning and random binning to the case when side information is present at the decoder. For 
ease of exposition and understanding in the following section, we first describe a lattice coding strategy for 
the distributed source coding problem involving two sources with the goal of reconstruction of their linear 
function at the decoder and, in addition, the decoder has access to some side information. We then use this 
to formally describe an achievable rate region for the problem of reconstructing Z = X^iLi CiXi. 



4.1 Lattice coding in presence of decoder side information 

In this section, we consider the problem of distributed encoding of correlated sources using lattices in the 
presence of side information at the decoder. As we will see, this can be used as a building block in recon- 
structing a linear function of multiple sources. 

Assume that we have correlated Gaussian sources Xi and X2 and the decoder is interested in reconstruct- 
ing a linear function Z = Ci^i. Suppose the decoder also has available to it side information Y that is 
correlated with the sources Xi,X2. Y and Xi,X2 are jointly Gaussian. Each source Xi is observed by an 
encoder which maps its outcomes to a finite set. The indices produced by the encoders are transmitted to a 
joint decoder using two rate-constrained noiseless channels. The goal is to find the optimal rate-distortion 
region which is the set of all achievable tuples (i?i ,R2,D). 

In this subsection we provide an inner bound to the optimal rate-distortion region for this problem using 
a lattice-based "correlated" binning strategy. We use the notation Zy to denote the minimum mean-squared 
error (MMSE) estimate of Z given Y, namely E{Z \ Y). The innovations random variable Z — Zy is denoted 
by VziY- 
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The lattice coding strategy in the presence of side information can be inferred by considering what the 
strategy would be in the presence of a central encoder that has access to all the sources Xi , X2 and the 
side information Y. In that case, the central encoder would first compute Z = CiXi and then quantize 

and transmit only the innovations random variable riz\Y- This can be accomplished with subtractive dither 
lattice quantization using a nested lattice A2 C Ai of parameter 



D 

4 



-'(Al) - (51) 



't'(A2) = (52) 
0-2 - D 

where cr^ is the variance of the innovations random variable riz\Y and D is the desired distortion in the 
reconstruction of Z. The rate incurred in this system is given by ^log{a^/D). The decoder would use this 
quantized innovations with the side information to obtain a reconstruction that is within a distortion of D 
of Z. 

The two assumptions in the setup above that deviate from our distributed coding problem are that all 
sources are available to a central encoder and that side information is available at the encoder. The first 
assumption can be gotten rid of by employing the distributive property (equation of lattice codes. The 
second assumption can be eliminated by using the linear nature of the forward test channel for the case of 
Gaussian quantization. This linear nature enables one to move the side information present at the encoder 
to the decoder thus obviating its necessity at the encoder. Thus, we can convert the above centralized coding 
strategy to our distributed setting to yield the following encoding scheme. 

The source encoders are described by the equations 

S^ = (gA,,(c,Xr + U^)) mod A2 for I = 1, 2, (53) 

where UiS are independent random dithers uniformly distributed over the fundamental Voronoi region Vo,ii 
of the fine lattices Ai^s. As in Section [3] we require A2 C Aii, i — 1,2, the fine lattices Aij to be good source 
codes and the coarse lattice A2 to be a good channel code. The second moments of the nested lattices are 
given by 

fT2(Aii) = qi (54) 
-(Ai2)=-^-9i (55) 

-'(A2) = (56) 

where qi is chosen such that < gi < ^2„'^ • This gives a quantization rate of 

1. ^4 



^^^o log W-2 h ^ (58) 

2 Da^-qi{a^-D) 



15 



Clearly, for a fixed choice of qi all rates beyond that given above can be achieved. Eliminating qi between 
the two rates now gives us an expression of the overall achievable region as 

-1 



-2Ri 



-2R2 




The decoder is given by the equation 




J2lA - u,) - Zf 



mod 



(59) 



(60) 



The encoding operation given by equation (53) is similar to that used in Section 3.2 By mimicking 
we can show that the first part of the decoder operation, given by (Ei=i('S'i ~ 



3.2 



the analysis of Section 

Ui) — Zy] mod A2) in equation (60), produces with high probability + N where N approaches a white 

— Da^ 

Gaussian noise vector with each element having variance o'^(Aii) + o'^(Ai2) = 2_'U ■ The decoder then 



obtains an estimate of the function Z based on 



Vz\Y 



N and the side information Y. It can be checked 



that equation ( 60 1 describes such an estimate and that this estimate indeed achieves the desired distortion 



D. Thus, we have an achievable rate-distortion tuple given by equation (59 1 for reconstructing a linear 



function in the presence of any side information. The rationale for choosing the lattice parameters and 



scaling constants is very similar to that given in Section 3.3 



4.2 Reconstructing a linear function of K sources 

Previously, we considered the problem of reconstructing a linear function of two sources. In this section, 
we generalize the problem to an arbitrary number of sources. Let the sources be given by Xi,X2, . . . , 
which are jointly Gaussian. The encoder of Xi maps its outcome to a finite set. The output of the encoder is 
transmitted over a noiseless but rate-constrained channel to a joint decoder. The rate of channel i is given by 
Ri. The decoder wishes to reconstruct a linear function given by Z — X^i^i Ci^i with squared error fidelity 
criterion. The performance limit TZV is given by the set of all rate-distortion tuples {Ri,R2, . . . ,Rk,D) 
that are achievable in the sense defined in Section [3] In this section we provide an inner bound based on 
"correlated" lattice-structured binning. 

As indicated earlier, there are several possible coding schemes based on each user's choice of coding 
strategy and also the choice of order of decoding. Before, we describe these coding schemes, we introduce 
some relevant notation. 

For any set Ac {!,..., K}, let Xa denote those sources whose indices arc in A, i.e., Xa — {Xi : i E A}. 
Let Za be defined as J^ieA ^iXi. Let be a partition of {1, ... , A'} with 6 = |G|. Let 7re : 6 — > {1, ■ ■ ■ ,9} 
be a permutation. One can think of ttq as ordering the elements of O. Each set of sources Xa, A G Q are 
decoded simultaneously at the decoder with the objective of reconstructing Za- The order of decoding is 
given by '7Tq{A) with the lower ranked sets of sources decoded earlier. Let Q= {qi, . . . ,qK) & be a tuple 
of positive reals. Let E(-) denote the expectation operator. 

For any partition O and ordering ttq, let us define recursively a positive- valued function CTq : O ^ as 
follows: 

aliA) = E[{ZA-fA{SA)f], (61) 
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where 



fAiSA)=EiZA\SA) 

Sa = {Zb + Qb ■■ B e e,7re{B) < TTeiA)} 



(62) 
(63) 



and {Qa '■ A G 0} is a collection of |0| independent zero-mean Gaussian random variables with variances 
given by qA — Var((5y!i) — X^iga ^^^^ collection is independent of the sources. Let 



f{{ZA + : ^ e e}) ^ E {Z\{Za + : a e 6}) 



Theorem 4.1. For a given tuple of sources Xi, . . . , Xk and tuple of real numbers (ci, C2, 
Ti-V*^ C TZV, where 

al{A) + qA 



(64) 

, ck), we have 



nV,„^ y 1{R,,...,R^,D): R,>hog^^^^^^ forieA 



i^>E[(Z-/({ZA + QA : Aee}))'] }, (65) 

and * denotes convex closure. 

Proof: We give a description of a lattice-based coding scheme that achieves the inner bound. Fix 0, 
TTe and Q. For each A E Q, construct a family of good nested lattices A^^ and Af such that C A^^ for 
i G A. Existence of such good nested lattices has been shown in Appendix [B] The second moment of the fine 
lattice A^j is chosen to be qi. The second moment of the coarse lattice is chosen based on the amount of side 
information available to the decoder at the time of decoding the set of sources Xa which in turn depends 
on TTe (A). The function erg, governs this choice. More precisely, for i S A and A E Q, the second moments 
of the lattices are given by 



(66) 
(67) 



Encoder: For each A £ Q, the source Xi, i e A is encoded using the nested lattice A^ C A^^. The 
encoders can be described by the equations 



T, = (Qa^. (cjXf + Ui)) mod A^ for i e A 



(68) 



where Ui are independent random dithers uniformly distributed over the fundamental Voronoi region 
of the fine lattice A^^. This would give an encoding rate of 

al{A)+qA 



Ri 



1 



;10g- 



for i £ A 



(69) 



2 - % 

Decoder: For A e 0, in order to decode Za, the decoder has access to some side information and its 



operation can be recursively described similar to equations ( 30 1 and ( 60 1 as 



Za^ 



J2m - u,) ~ f\{SA) 



mod A^ + f\{SA) 



(70) 
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where 

SA^{ZB:Bee,Tre{B)<7re{A)}. (71) 

After decoding for all A E Q, the decoder obtains the reconstruction as a hnear function of {Z4 : A E Q} 
as 

z^razA-.Aee}). (72) 



We now show that the above system achieves the inner bound given in the theorem. From equation ( 69 1 , it 



is clear that this scheme achieves the rate tuple claimed in Theorem 4.1 It remains to prove that the claimed 



distortion is achieved. The crucial observation is that while Sa in equation (63 1 denotes the side information 



available to decode Za in test channels, Sa in equation (71 ) denotes the side information available to decode 
Za in the actual coding system. If we were to assume Sa to be Gaussian, then by definition of the functions 

is 



/a(-) (equation (62l) and /(•) (equation (64l), it is easy to see that the distortion given in Theorem 4.1 
achieved. However such an assumption isn't true for Sa for any finite lattice dimension n. 

Fortunately, loosely speaking, we can show that even though the assumption of Gaussianity of Za isn't 
strictly true, it becomes increasingly valid as the lattice dimension n —^ 00. By analysis similar to that in 
Section |3.2| we can show that the subtractive dither quantization noises tend to a white Gaussian of the 
same variance (in the K-L divergence sense). This implies that as the lattice dimension n ^ 00, for an 
optimal choice of nested lattices, Za tends to Z^ + and hence Sa tends to S^ (in the K-L divergence 
sense). By virtue of the "goodness" of the nested lattices, this then implies that the probability of incorrect 
decoding goes to exponentially in the lattice dimension. Thus the reconstruction error (Z" — Z) tends in 
probability (and hence in normalized second moment) to N where N approaches a Gaussian random vector 
with each component having variance D. Thus, the proposed lattice scheme indeed achieves the claimed 
rate-distortion tuples and Theorem |4.1| is proved. 

To show this formally using induction, we need some more notation. For each A E Q and for each i E A, 

let 

e, = Qj,A (cXf + U,)- c,Xf - , (73) 

and 

e^^^e,. (74) 
For each A G O, let the linear function /a(-) be given by 

fA{SA)= aA{B){ZB + QB)- (75) 

B:ire{B)<7re{A) 

By noting that are independent for i £ {1,2,..., K}, we note that for all A £ Q, 

1, 



n^Af = QA. (76) 



n 



Let E G Q he such that ttb{E) ~ 1. Thus Se = <!>■ Hence using the distributive property, and noting the 
normalized second moments of for i £ E, we have with high probability (i.e., under correct decoding) 

ZE = Z'^ + eE. (77) 



18 



For any 1 < j < i^T, we assume correct decoding with high probabiUty at the jth stage and show correct 
decoding with high probabihty at the (j + l)th stage. Let C S G be such that 7re(C) — j + 1. Under the 
above assumption, we have, with high probability, for all i? e 6 with ttq < j 



Using this we have 



Zb — Zg + cb ■ 



(78) 



c.d 



Zc + ec- (^c{B)Zb\ mod ^ ac{B)Zi 

S:7re(-B)<j / B:7re(S)<j 

Zc + ^c- ^c{B)Zb + ^c{B)Zb 

B:TTe{B)<j J B:TTe{B)<j 



(79) 

(80) 
(81) 



where the second equality holds with high probability (correct decoding) because of the following reasons, 
(a) The normalized second moment of the term inside the mod operation satisfies the following equalities: 

2 



Zc + ec- Y ^c{B)Zb 

B:-KeiB)<j 



(82) 



-E 



- Y '^c{B)Zl 

B:-Ke{B)<] 



qc+ Y (^c{B)qB 
B:7re(B)<j 

gc + fiUc- Y ac{B){ZB + QB) 

\ B:ne(B)<j 
al{C) + qc 



(b) Using the arguments of Section 3.2 (see Appendix O), 



lim/i Zg + ec- V ac{B)ZB\ 

\ B:^e{B)<j J 



log27re(72(AC^) 



(83) 

(84) 

(85) 
(86) 



(87) 



where h{-) denotes differential entropy. Hence we have for all A £ Q, with high probability, 

Za — Z^A + ^A- 

Now regarding the final estimation, an argument similar to the above can be given that shows that a 
distortion given in the theorem is achieved asymptotically. The rationale for the specific choice of scaling 
constants is explained in detail in Appendix [P] 



Remark: An important point worth noting before proceeding further is that the nesting relations we need 
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the lattices to satisfy is C A^j for i £ A. But, for A,B £ Q, we don't need the lattice families (A^j, A^) 
and (A;[^ , A^) to be related in any way for A ^ B. Also, just as in the two user case, if we are interested 
only in minimizing the sum rate of this encoding scheme, then for all encoders in a given set A G O, the 
second moment of their respective fine lattices are equal. This means that all encoders in a given set A E Q 
can use the same nested lattice A^ C A^ for encoding. 



4.3 An illustration of Theorem 14.11 

For clarity, an illustration of the coding scheme of Theorem |4.1| for the case of 6 users and specific choices of 
Q and ttq is described below. Let us choose Q = {{1, 2, 3}, {4, 5}, {6}}. Let ttq be the identity permutation 
so that 7re({l,2,3}) = l,7re({4, 5}) = 2,7re({6}) = 3. This means that the decoder decodes ^{1.2,3} = 
Si=i Ci^i fii'st which is then used as side information for decoding •2^(4,5} and so on. Let us also fix 
Q = {qi, . . . ,qQ} where qt are all positive. We use A,B,C to denote the sets {1, 2, 3}, {4, 5} and {6} 
respectively. 



The fine lattice of the encoder of source Xi has second moment qi as given in equation ( 66 1 . Encoders 



for the sources Xi , X2 , X3 use nested lattices where the second moment of the coarse lattices are given by 



equation (67 1. The decoder decodes Za according to equation (70 1. To decode Za, the decoder does not 
have access to any side information. Encoders for X^^X^ use nested lattices whose parameters depend on 
the function (Tq(-B) which in turn is determined by the fact that Za has been decoded earlier. The decoder 
then decodes Zb from T^^T^ and the functional value /]^(-) of the side information Sb = Za- Similarly, to 
decode Zc, the decoder has side information Sc — {Za, Zb} along with the index Tg. After having decoded 
Za, Zb,Zc, the decoder uses the function /"(•) of equation (64) to estimate Z. This is illustrated in Fig. IZ] 



4.4 A Few Special Cases 

In this section, we consider a few special cases of the general coding problem treated above. In particular, we 
examine the rate distortion region derived above for specific choices of the partition 0. First, we demonstrate 
that we can recover the two user rate region of Theorems |3.1| and |3.2| from the more general K-user rate 
region described above. Then, we illustrate a scheme for the case where the decoder estimates the function 
directly, i.e., 9 = {{1,2, . . .,K}}. 



4.4.1 Berger Tung coding for the two user case 



In this section, we rederive the result of Theorem |3 . 2| using the more general framework of Theorem |4.1[ Let 
the function to be reconstructed he Z — Xi — 0X2 as in Section |3] Individual reconstruction of the sources 
corresponds to the partition = {{!}, {2}}. There are two possible choices of vre corresponding to which 
source is decoded first. Let us choose ttq to be the identity permutation. Thus -^{i} — Xi is decoded first 
and used as side information to decode ^{2} = —cX2. 

Let Q — {qi, (72) where qi are positive for i — 1,2. For ease of notation, we drop the set notation in the 
subscripts below. In what follows. Si is taken to mean S^ij and so on. Equations (61) to (63 1 simplify in 
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Figure 4: Illustration of the coding scheme of Theorem |4.1 



this case to 



5i = 
=E(Zi) = 
4({1}) = E(Z?) = 1 
S2 = {Xi + Qi} 



pc 



h{S2) = E(Z2 I ^2) = E(-cX2 I Xi + Qi) = -y— 52 
4({2}) = E (^2 + ^^2)' = + 92 



(89) 
(90) 
(91) 
(92) 
(93) 

(94) 



gi y " 1 + 91 

It follows from estimation theory that the function f{Zi + Qi., Z2 + Q2) = o-{Zi + Qi) + 6(^2 + Q2) where 
the constants a, b are given by 



a b 



ac^+q2(i—pc) e(ae+gi(e— p)) 



(l + 9l)(c2+g2)-p2c2 (l + ,i)(c2+g2)_p2 



(95) 



where a = 1 — p^. 

As stated in Theorem 4.1 qi have to satisfy the distortion constraint of equation (651 which in this case 
simplifies to 



^ ^ qic^a + g2C^a + gi 92171 
- (l + 9i)(c2 + 92)-/o2c2 



(96) 
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The parameters of the nested lattices are given by equations ( 66 1 and ( 67 1 to be 



a'{Al'^)^c' + q2- 



1 + gi' 



This gives the following rates. 



2 qi 

1 (c2- 
2log — 



91 



i?2 



g2)(i + gi) -p'c^ 
92(1 + 91) 



(97) 
(98) 
(99) 

(100) 

(101) 
(102) 



where Q — (91,92) is subject to the distortion constraint of equation (96). It can be checked that these 



equations parameterize one of the corner points of the rate region of Theorem 3.2 Reversing the roles of the 
two sources (equivalently, choosing 7re({l}) — 2,7re({2}) = 1), we can achieve the other end point of the 
rate region. Time sharing between these two points achieves the entire rate region of Theorem |3.2| 



Note that the inner bound of Theorem 3.2 is derived using the Berger-Tung inner bound [SI |7] which 
employs random quantization followed by random binning. Here, we have rederived this result using lattice 
quantization followed by lattice-structured binning. 



4.4.2 Lattice coding for the K user case 

In this section, we derive an achievable rate region for the K user case when all the users encode in such 
a way that the decoder estimates the function directly without reconstructing any intermediate variables. 
This corresponds to the case where Q = {{1, . . . , -fC}}. Tre is trivial in this case. Let Q — {gi, . . . , g^} £ 



if. 



Let A denote the set {1, . . . , K}. Then q^ — X^iLi 9i 



Equations (61 1 to (63 1 simplify in this case to 



fAiSA) - nz) - 

4(A)=E(Z2) = a|. 



The function /(•) of equation (64 1 is given by 



/(Z + Q) = E(Z \Z + Q) 



-iZ + Q) 



and thus distortion constraint of equation ( 65 ) fixes the value of to be ^'2^_^ 



(103) 

(104) 
(105) 



(106) 
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The encoders use the nested lattices (Aii,A2),i = 1,. .. ,K for encodmg. The parameters of the nested 
lattices are given by 



This gives an encoding rate of 



cr^(Aii) = q. 



(107) 
(108) 



This corresponds to the rate region 



2 °(z,(ai-i?) 



K 



E2 



For K — 2^ this recovers the rate region of Theorem 3.1 



(109) 



(110) 



5 Comparison of the Rate Regions 

In this section, we compare the rate regions of the lattice based coding scheme given in Theorem |3.1| and 



the Berger-Tung based coding scheme given in Theorem 3.2 for the case of two users. The function under 
consideration is Z ~ Xi — 0X2- We would like to emphasize that we have assumed that the sources have 
unit variance and that p > 0. To demonstrate the performance of the lattice binning scheme, we choose the 
sum rate of the two encoders as the performance metric. 

Fig. [5] shows the sum rate of the lattice based scheme for different values of c and distortion D. In 
Fig. |6] we compare the sum-rates of the two schemes for p = 0.8 and c = 0.8. Fig. |6] shows that for small 
distortion values, the lattice scheme achieves a smaller sum rate than the Berger-Tung based scheme. This 
shows that the rate region of Theorem |3.1| contains points outside that of the rate region of Theorem |3.2[ 
The opposite is also true since for D = a-|, the region in Theorem 3.2 contains the rate point (0,0) while 
the one in Theorem 13.11 does not. 



We observe that the lattice based scheme performs better than the Berger-Tung based scheme for small 
distortions provided p is sufficiently high and c lies in a certain interval. Fig. [7] is a contour plot that 
illustrates this in detail. The contour labeled R encloses that region in which the pair (p, c) should lie for the 
lattice binning scheme to achieve a sum rate that is at least R units less than the sum rate of the Berger-Tung 
scheme for some distortion D. Observe that we get improvements only for c > 0. 



6 Conclusion 

We have thus demonstrated a lattice based coding scheme that directly encodes the linear function that 
the decoder is interested in instead of encoding the sources separately and estimating the function at the 
decoder. For the case of two users, it is seen that the lattice based coding scheme gives a lower sum-rate for 
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Sum rate vs c and D for rho = 0.8 




Figure 5: Lattice based scheme's sum-rate vs c and distortion D ior p = 0.8 




Figure 6: Comparison of the sum-rates of the 2 schemes 
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Region where lattice sctieme outperforms Berger-Tung schieme 
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Figure 7: Range of (p, c) where the lattice scheme performs better than the Berger Tung scheme 

certain values of p, c, D. Hence, using a combination of the lattice based and the Berger-Tung based coding 
schemes results in a better rate-region than using any one scheme alone. For the case of reconstructing 
a linear function of K sources, we have extended this concept to provide an inner bound to the optimal 
rate-distortion function. Some parts of the inner bound are achieved using a coding scheme that has the 
following structure: lattice vector quantization followed by "correlated" lattice-structured binning. 
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A Derivation of Berger-Tung based scheme's sum rate 



In this section, we derive the sum-rate of the Berger-Tung based scheme given in equations (21)-(23l. The 
sum-rate of the Berger-Tung based coding scheme is given by 



«, + fl,>ilog<i±Ml±<5)^ 

2 qiq2 



(111) 



where ((71,92) G IR+ should satisfy the distortion constraint 



„ gict + q2C^a + qiq2(Tz 
- (l + 9i)(l + g2)-p2 



(112) 
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where IR+ is the set of positive reals and a — 1 — 



To minimize the sum-rate, we need to minimize the quantity given by equation (111 I. Using the fact 



that the log function is monotone and that ((71,52) must satisfy the distortion constraint in equation (1121, 
the minimization problem is equivalent to minimizing 



(1 + gi)(l + (72) - _ 91" + <72C^a + qiq2<7% 



9192 



Dqiq2 



and this is equivalent to minimizing 



1 

— H 

92 91 



(113) 



(114) 



subject to the constraint in equation (112). 

Assuming that (51,(72) satisfy the distortion constraint with equality, one can solve for (72 in terms of qi 
to give 

aD — qi(a — D) 



92 



{c-^a-D)+q^{al-Dy 



(115) 



Substituting this in equation ( 114 1 gives the function to be minimized as a function of qi alone. The optimal 
choice of qi is then 

ql{(jl - D)+ qiD{c^ - 1) + aDc^ 



Qi = argmm 



—qf{a — D) + aDqi 



(116) 



Differentiating with respect to qi and setting the derivative to gives us a quadratic in qi whose roots are 



acD 



9i 



or 



(117) 



p~c 2ac—{p + c)D 

The second root given above is where the minima occurs. The (72 value corresponding to this value of qi is 

aD 
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(118) 



2q;c2 - (1 + pc)D' 

Note that these optimal values of qi and 52 are positive only for distortions in the range 

D<n{in{^,-—\. (119) 

yp + c 1 + pc ) 

For values of D outside this range, the optimal strategy is to let qi or (72 go to 00 which effectively means 
that we encode and transmit only one source. 

For D in the range given in equation (1191, the sum rate i?suin = Ri + R2 is found by substituting q^ 



and (72 in equation (111) to get 



Rs 



> -l0£ 



Ac^ac — pD) 



D < min 



2ac 2oL(? 



p + c 1 + pc 



(120) 



For D outside the range given in equation (119), the minimum sum rate is attained by setting either qi or 



(72 as cx). Which quantity goes to 00 depends on which argument of the min function in equation (119) is 
smaller; equivalently on whether c > 1 or not. It is easy to see that if c < 1, (72 — cxd and 



1 il-pcf 2ac^ 

Rsu.n = log ^ > ' 

2 U — ac/ I + pc 



(121) 
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and if c > 1, gi = CX3 and 



1. (c-p)2 2ac 

-los- ^ for D > . 

2 D~a p + c 



(122) 



Combining equations (1201, (1211 and (122) and taking the convex closure of the resulting region, the 
complete rate region for the Berger-Tung based scheme can be found. 



B Existence of good nested lattices 

In this section, we show the existence of nested lattices with any finite degree of nesting such that all 
the lattice codes involved are simultaneously good source and channel codes. More precisely, we show the 
existence of a nested lattice (Ai, . . . , A^), A^ C • • • C Ai such that A.^, z = 1, . . . , m are simultaneously good 
source and channel codes for sufficiently large lattice dimension n. 

We use the same nested lattice ensemble as described in [JB]. For completeness sake, we include a 
description of the ensemble. 

• Start with a fixed n-dimensional lattice A2 which is a good source and channel code. The existence of 
such a lattice was shown in [?7j. Let be the generator matrix of A2, i.e., A2 = ■ 

• Construct a, k x n matrix G whose elements are drawn according to an uniform i.i.d distribution over 
Zp = {0,l,...,p— 1} where p is an appropriately chosen prime. 

• Define the discrete codebook C = {x E : x = y ■ G ior some y E Z^}. 

• Apply Loeliger's type A construction 52J to form the lattice Ai = p^^C + I/\ 

• Transform to get the fine lattice Ai = Ga2 • A^. 

• This construction of Ai can be viewed equivalently as follows. From the lattice p~^A2, pick k < n 
points at random along with all their multiples modulo- A2. The resulting set of points constitute the 
fine lattice Ai. 

By construction, it follows that A2 C Ai with the nesting ratio \/p^. It was shown in [46^ that, with high 
probability, this construction will result in Ai being a good channel code. In [53 , it was shown that a nested 
lattice from this ensemble is, with high probability, a good source code as well. By union bound, it then 
follows that Ai is simultaneously a good source and channel code with high probability. It follows that there 
exists nested lattices (Ai,A2) such that A2 C Ai and both Ai and A2 are simultaneously good source and 
channel codes. 

By iterating this process (with Ai playing the role of A2 in the construction above), one can obtain a 
nested lattice code with any finite level of nesting. More precisely, for any to > 0, one can show the existence 
of a nested lattice (Ai, . . . , A„j), A„j C • • • C Ai such that all the lattices Ai,i = 1, . . . , m are simultaneously 
good source and channel codes. Moreover, this can be accomplished for any choice of nesting ratios. 
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C Proof of convergence to Gaussianity of 

In this section, we prove the claim that = e^^ — tends to a white Gaussian noise in the KuUback-Leibler 
divergence sense. Note that e^^ and e^^ are independent. 

We use the following properties of subtractive dither quantization noise and the associated optimal lattice 
quantizers [42] . 

• The subtractive dither quantization noise e^. is uniformly distributed over the basic Voronoi region 
Vo.ii of the fine lattice Ai^ for i = 1, 2. It follows from equation ([s]) that 

E II eq^ |p= na'^{Ku) for i = 1,2. (123) 

• For optimal lattice quantizers, the components of eq-,i — 1,2 are uncorrelated and have the same 
power, i.e., their correlation matrices Sg^ can be written as 

Ee,^ = a2(AH)I„x„ for i = 1, 2. (124) 

• For optimal lattice quantizers, as the lattice dimension n ^ oo, the distribution of e^. ,i — 1,2 tends 
to a white Gaussian vector of same covariance in the KuUback-Leibler divergence sense. Taking into 



account equation (123), this can be written as 

^Z?(e,. II AA(0,a2(AH)I„x„)) ^ for ^= 1,2 (125) 
in terms of the KuUback-Leibler divergence D{. \\ .) or equivalently, 

/i(egj ^log27reCT^(An) fori = 1,2 (126) 
in terms of differential entropy h{-). 

To show the convergence of to a white Gaussian random vector, we use the entropy power inequality 
and the fact that for a given covariance matrix, the Gaussian distribution maximizes differential entropy. 

The entropy power inequality |13] states that for two independent n-dimensional random vectors X and 
Y (having densities), 

2^h(x+Y) > 2lh(x) ^ r^lKY)_ (127) 

This inequality applied to the subtractive dither quantization noises gives 

2 i ''(^91 "^92) > 2"''('^9i) -I- 2"''^^«2). (128) 



As n ^ 00, by equation (126), the right hand side of equation (128) tends to 27re(o'^(Aii) -f ct^(Ai2)). So 



we have the following lower bound on the limit of the differential entropy of e^. 



n 



lim h{eq) > -log27re(CT^(Aii) -|-(7^(Ai2)). (129) 
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To prove the inequality in the other direction, note that equation (1241 imphes that the covariance matrix 
of Cq is (cr^(Aii) + a^(Ai2))Inxn- Since the Gaussian distribution maximizes differential entropy for a given 
covariance matrix, we have 



h{eq) < -log27re(a2(An)+CT2(Ai2)) 



(130) 



Combining equations (1291 and ( |130[ ), we have the desired result that (if optimal lattice quantizers are 
used) 



lim h{eq) = -log27re(CT2(An) +(7^(Ai2)). 



(131) 



In words, Cq tends in the KuUback-Leibler divergence sense to a white Gaussian random vector with covari- 
ance matrix (o-2(Aii) + cr^(Ai2))I„xri- 



D Derivation of optimal Lattice parameters 

In the coding schemes of both Section [3] and Section |4] we scale the sources before encoding them. Here, we 
briefly outline a justification for the specific scaling constants used. We restrict ourselves to the case where 



all the K users encode their sources using lattice binning. In the notation of Section 4.2 this corresponds 
to e = {l,...,K}. 

Let the function to be reconstructed he Z ^ — cX". Here c is a row vector with its ith 

component as Ci and X" is a column vector of the sources Xi. S is the covaraince matrix of the random 
vector X". Let the ith encoder scale its input by an arbitrary constant rji. Let 77 = [771, ... , r/x]- Choose a 
tuple Q— {qi, . . . ^qx) G just as in Section 



4.4.2 



and the corresponding distortion is 



It can be shown from analysis similar to the ones in Section |3.2| and |4.2| that the decoder can, with 
high probability, reconstruct the function rjX" + Q where Q approaches a white Gaussian noise of variance 
q = Qi- From equation (64 1, it follows that the function / used for decoding is 

Z= ( ff^ ){VX'' + Q) (132) 

^ = <^z ^r^. — • (133) 

77^77-' + q 

This fixes the value of q. The second moment of the channel code used is cr^(A2) = Var(^^ rjiXi + qi) — 
•qTiT]^ + q. This gives us the rate tuple 

i?,^ ^log + g fori = l,...,K (134) 

2 qi 

Eliminating qi using q — qi gives us the rate region 



2-"'- <l-(^i-D)^^§^^ (135) 



This rate region is largest when the RHS is maximum. Maximizing the RHS as a function of 77 results in 
77 = ^ • c as the only solution for some constant ^. However, all constants ^ result in the same rate region. 
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